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I.  Introduction 

o  The  scope  of  this  study  probably  deserves  more  resource  and  time  to  gain 
in-depth  insight.  However,  INPUTs  extensive  research  in  productivity 
improvement  in  systems  and  software  implementation  will  permit  us  to 
place  the  specific  research  associated  with  this  study  into  a  broader 
•framework  which  will  make  it  more  meaning-ful. 

o  One  observation  o-f  importance  should  be  made  at  the  beginning:  it  is 
INPUTs  opinion  that  the  tools  requ  i  red  -for  the  development  o-f  business 
systems,  and  those  -for  the  development  o-f  embedded  and/or  process  control 
systems,  are  becoming  indistinguishable  as  computer/communications 
networks  permit  <and  often  -force)  levels  o-f  integration  where  the  -flow  o-f 
business  data  and  i  n-f  orrnat  i  on  becomes  an  essential  process  to  the 
enterprise.  It  is  also  true  that  productivity  tools  must  assist  in  the 
development  o-f  applications  systems  which  will  be  spread  over  the 
processing  hierarchy  (main-frames,  minicomputers,  and  m i crorocessors)  and 
be  dynamic  in  the  allocation  o-f  these  resources.  The  -fact  that  adequate 
tools  do  not  currently  exist  to  address  these  problems  is  primarily  the 
result  o-f  inadequate  problem  definition  rather  than  technical   in  nature. 

o  The  research  approach  INPUT  has  taken  for  this  study  is  to  rely  heavily 
upon  past  detailed  research,  and  use  telephone  interviews  with 
recognized  experts  to  update  the  current  status  of  both  specific  case 
studies  involving  the  use  of  productivity  tools  and  aids,  and  to  refine 
general  patterns  of  acceptance  and  usage  of  tools  and  aids,  and  to  verify 
our  analysis  of  the  software  development  productivity  problem. 

o  The  major  productivity  research  studies  which  have  been  conducted  by 
INPUT  will  be  referenced  in  this  report.    They  are  as  follows: 
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The  seminal  productivity  study  was  sponsored  on  a  multiclient  basis 
by  some  o-f  the  largest  corporations  in  America  and  consisted  o-f  the 
f  ol  1  ow  i  ng : 

In-depth,  on-site  interviews  were  conducted  at  multiple  staff 
levels  (programmer/analyst  through  MIS  Director)  in  60  -firms  to 
determine  their  approaches  to  the  improvement  o-f  software 
development  and  maintenance. 

An  additional  32  organizations  which  had  enmployed  speci-fic 
tools,  approaches  and  methodologies  were  interviewed  over  the 
telephone  to  complement  the  on-site  interviews. 

Approximately  1,300  mail  surveys  were  conducted  to  provide  a 
statistical  base  -for  eveluating  the  results  o-f  the  in-depth 
interviews. 

Extensive  desk  research  was  conducted  on  published  i  n-f  ormat  i  on  . 

When  warranted,  authors  and  experts  who  have  made  speci-fic 
contributions  to  productivity  improvement  were  interviewed 
either  in  person  or  on  the  telephone. 

The  results  o-f  this  study  was  presented  to  clients  in  a  series 
o-f  seminars,  and  a  400  page  report  o-f  -findings  was  published  by 
INPUT  in  1980.  (Improving  the  Productivity  o-f  Systems  and 
So-ftware  Implementation) 

In  1982,  INPUT  conducted  a  care-fully  selective  updating  and 
extension    of    Improving    the    Productivity    o-f    Systems    and  Software 
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Impl ementat  i  on  -for  the  Computer  Development  Laboratories  <CDL). 
This  study  concentrated  on  the  -following: 

On-site  interviews  with  knowledgable  experts  in  seven  broad 
industry  segments. 

On-site  interviews  with  -five  vendors  advocating  the  -following 
approaches  to  productivity  improvement:  4GL/DBMS,  applications 
packages,  information  management  systems< data  management, 
statistical  analysis,  and  i  n-f  orrnat  i  on  <graph  i  cs>  display 
systems),  custom  systems  consulting  and  implementation 
services,  and  data  base  machines. 

In  addition,  special  attention  was  given  to  the  work  being  done 
in  Software  Engineering  Economics  (Barry  W.Boehm),  Information 
Engineering  <James  Martin  and  Clive  F i nk 1 este i n ) ,  and  Queuing 
Networks  <Dr.  Ralph  L.  Disney). 

The  results  of  this  study  was  presented  to  CDL  in  a  200  page 
report  -  Software  Development  Productivity;  INPUT,  December 
1982. 

In  1984,  as  part  of  its  continuing  subscription  programs  for 
computer  users  and  vendors  INPUT  published  two  companion  reports  on 
the  use  of  productivity  tools,  techniques  and  methodologies. 

New  Opportunities  for  Software  Productivity  Improvement  was 
addressed  to  users  and  focused  on  the  the  emerging  systems 
development  environment  for  business  systems  and  potential 
problems  associated  with  that  environment. 
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Market  Impact  o-f  New  Software  Productivity  Techniques  contained 
specific  recommendations  -for  vendors  -for  new  productivity  tools 
required  to  address  the  anticipated  problems  o-f  the  emerging 
systems  development  environment. 

In  early  1985,  INPUT  had  an  opportunity  to  address  the  primary  issue 
which  was  isolated  in  the  1984  reports  -  quality  assurance.  This 
study  was  conducted  -for  JIPDEC  and  included  specific  case  studies 
studies  o-f  what  was  being  done  in  selected  organizations.  Both  on 
site  and  telephone  interviews  were  conducted  and  the  title  o-f  the 
report  was  So-ftware  Quality  Assurance. 

Also  in  1985,  INPUT  reviewed  the  current  status  o-f  knowledge-based 
systems  (expert  systems),  as  a  potential  tool  -for  productivity 
improvement.  The  resulting  report  -  Arti-ficial  Intelligence  and 
Expert  Systems  was  published  as  part  o-f  INPUTs  subscription  program 
■for  computer  users. 

o  Analysis  of  individual  so-ftware  development  tools  has  not  been  attempted 
except  to  classify  them  based  upon  their  stated  purpose  and  how  they  have 
been  employed.  INPUT  urges  caution  in  accepting  published  information 
concerning  productivity  improvement  based  on  either  vendor  claims  or  as 
the  result  of  case  studies  which  have  not  been  thoroughly  researched. 
There  are  two  reasons  for  this:  1)  valid  metrics  are  seldom  employed  to 
support  such  claims,  and  2)  the  classic  "Hawthorne  effect"  can  be 
observed  when  new  tools  are  introduced  into  an  organization  on  a  trial 
bas  i  s . 
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o  No  attempt  to  summarize  the  in-formation  contained  in  this  study  will  be 
made  since  the  entire  report  can  be  considered  a  summary  of  extensive 
past  research.  It  is  strongly  recommended  that  a  more  targeted  research 
project  be  undertaken  i -f  the  purpose  o-f  this  study  is  to  be  used  -for 
either  applications  development  tool  selection  or  marketing. 
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II.  Review  of  Past  INPUT  Research 


o       The     primary    conclusions    reached     in     Improving     the     Productivity  of 
Systems  and  Software  Implementation  (INPUT,  1980)  were  as  -follows: 

Then  current  "solutions"  did  not  solve  productivity  problems.  The 
problems  were  manifested  by  the  following: 

Systems  were  becoming  more  complex  and  expensive,  and  time  and 
cost  overruns  were  becoming  even  more  of  a  problem  than  they 
had  once  been. 

Software  maintenance  was  consuming  ever  increasing  amounts  of 
the  software  dollar,  and  users  were  dissatisfied  with  quality 
and  re  1 i  ab  i 1  i  ty . 

Backlogs  continued  to  increase  while  software  related  expenses 
were  becoming  an  increasing  portion  of  total  systems  budgets. 

Managements  perception  of  the  problem  was  limited  to  a  narrow  focus 
on  schedule  overruns  and  the  increasing  backlog.  Increasing  costs 
and  cost  overruns  were  considered  "normal"  (or  secondary  issues), 
and  software  quality  was  rarely  cited  as  a  contributing  problem. 
The  result  was  undue  emphasis  upon  tools  (both  hardware  and 
software)  to  improve  programmer  productivity,  and  failure  to  develop 
a  consistant  strategy  for  productive  systems  development. 

The  major  INPUT  conclusion  was  that  an  overall  strategy  was  required 
and  that,  unless  certain  priorities  were  established,  environmental 
changes  and  "investing"  in  improved  tools  and  would  not  be 
effective.    This  conclusion  lead  to  the  "Productivity  Pyramid"  which 
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has  been  published  in  numerous  INPUT  reports.  Exhibit  II. 1. 
Essentially,  this  relatively  simple  diagram  emphasizes  the  need  to 
build  an  effective  strategy  -from  the  bottom  up,  and  it  warrants 
repetition  in  this  study. 

Commitment  to  quality  software  may  sound  simplistic,  but  there 
has  been  a  tendency  to  concentrate  on  the  latest  hardware 
technology  and  assume  that  necessary  software  can  always  be 
improved  later  -  the  result  has  been  a  lot  of  "quick  and  dirty 
software"  at  all   levels  in  the  processing  hierarchy. 

User  involvement  throughout  the  development  process  is 
absolutely  essential    if  a  quality  system  is  to  be  implemented. 

Broadbased  management  stresses  the  necessity  for  all  levels  of 
management  <for  example,  corporate  executives  and  engineering 
management)  to  recognize  the  importance  of  quality  software 
development  over  the  systems  life  cycle,  and  to  remain  involved 
in  the  establishment  of  priorities  and  the  allocation  of 
resources . 

Effective  personnel  can  be  defined,  selected  and  retained  only 
with  full  understanding  of  systems  complexity  and  associated 
pr  i  or  t  i  es . 

Only  after  the  first  four  levels  of  the  productivity  pyramid 
are  assured  can  the  r  i  qh  t  tool  s  be  selected;  and  it  must  be 
understood  that  there  is  no  one  right  set  of  tools  for  all 
projects  and  all  personnel. 
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EXHIBIT  11-1 
THE  PRODUCTIVITY  PYRAMID 
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While  the  importance  o-f  the  right  tools  was  placed  into  proper 
perspective  in  the  study,  this  should  not  be  regarded  as  a  lack  o-f 
concern  about  the  availability  o-f  appropriate  tools  and  INPUT 
emphasized  that  available  tools  usually  addressed  only  portions  o-f 
the  systems  life  cycle.    Exhibit  II. 2. 

o  In  view  o-f  the  general  statement  o-f  requirements  -for  the  current  study, 
it  should  be  pointed  out  that  sponsors  o-f  the  multiclient  study  included 
several  aerospace  companies  where  software  development  for  embedded, 
on-line,  real-time  computer  systems  have  long  been  of  primary  concern; 
and  where  the  most  comprehensive  work  in  attempting  to  improve  software 
quality  and  productivity  have  occurred.  An  aerospace  company  which 
represented  one  of  the  detailed  interview  sites  for  the  1980 
research<not  a  sponsor  of  the  study),  will  be  used  as  a  case  study  later 
In  this  report . 

o  The  1982  custom  study  Software  Development  Productivity  identified  major 
trend  towards  getting  end  users  involved  in  the  systems  development 
process,  and  speculated  that  this  was  the  direct  result  of  the 
proliferation  of  personal  computers  in  the  corporate  environment. 
Therefore,  information  systems  managers  were  using  this  involvement 
because  it  was  expedient  <as  opposed  to  any  commitment  to  quality),  and 
INPUT  foresaw  certain  problems  which  might  attend  this  approach  to 
productivity  improvement.    Specifically,  it  was  stated  that: 

User-developed  systems,  including  prototypes,  will  remain  installed 
and  require  maintenance  and  integration  with  other  systems. 
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EXHIBIT  11-2 


STAGES  OF  PRODUCTIVITY  IMPROVEMENT: 
COST/BENEFIT  RELATIONSHIPS,  STAGE  2 
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Data    and     information     bases    will     not     be    manageable     and  will 
proliferate  beyond  control. 

Data  processing  personnel  will  become  so  involved  with  the 
development  of  end  user  directed  systems  that  they  will  never  be 
able  to  consider  quality,  much  less  make  a  commitment  to  it. 
Getting  something  done  quickly,  regardless  of  quality,  will  be  the 
order  of  the  day. 

The    expense    of    systems    development    will     increase  substantially 
without  really  improving  productivity. 

o        In  addition,  the  1982  study  pointed  to  three  areas  which  seemed  promising 
to  the  overall  productivity  problem: 

The  work  of  Barry  W.  Boehm  as  presented  Software  Enoineerino 
Econom  i  cs  was  mentioned  as  holding  promise  for  solving  the 
continuing  problems  of  software  productivity  measurement  and  metrics 
-  prov  i  ded  this  work  was  used  intelligently.  <It  had  been  earlier 
determined  that  lack  of  accepted  metrics  was  a  significant 
impediment  to  progress  in  productivity  improvement.) 

Work  in  Information  Engineering,  as  espoused  by  James  Martin  and 
Clive  Finklestein  was  also  mentioned  as  exhibiting  some  promise  in 
the  critical  area  of  data  and  information  quality.  (It  should  be 
pointed  out  that  information  quality  is  an  essential  ingredient  of 
productivity  improvement  whether  one  is  considering  the  quality  and 
precision  of  internal  systems  interfaces  or  the  communications 
among/between  project  teams  and/or  individual  analysts  or 
programmers . ) 
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Work  on  Queuino  Networks  as  it  has  evolved  in  operations  research 
was  deemed  to  be  of  particular  importance  to  not  only  the  problems 
of  internal  performance  o-f  complex  systems  but  also  to  the  problems 
which  INPUT  -felt  were  inherent  in  project  team  commun  i  cat  i  ons ;  and 
an  on-site  interview  with  Dr.  Ralph  L.  Disney  o-f  Virginia 
Polytechnic  Institute  was  included  as  part  o-f  the  study. 

All  three  o-f  these  major  subjects  have  been  brie-fly  updated  as  part 
o-f  this  study. 

The  two  productivity  studies  conducted  by  INPUT  in  1984  (New 
Opportunities  -for  Software  Productivity  Improvement  and  Market  Impact  of 
New  Software  Productivity  Techniques)  confirmed  some  of  the  anticipated 
problems  identified  in  the  1982  study.  Essentially,  systems  were  being 
prototyped  using  applications  development  tools  (primarily  46Ls  and 
DBMSs) ,  and  applications  developed  in  information  centers  and  on  personal 
computers  were  being  evolved  into  production  systems  (or  becoming 
integrated  with  established  systems).  INPUT  refers  to  this  environment 
as  Distributed  Systems  Development  (DSD),  and  the  problems  specifically 
associated  with  this  environment  have  been  clearly  identified: 

The  synchronization  and  integrity  of  data  bases  in  the  DSD 
environment  is  a  technical  problem  which  does  not  have  an  accepted 
sol u  t  i  on . 

Security  and  protection  of  distributed  data  bases  adds  a  level  of 
complexity  to  a  problem  which  does  not  currently  have  accepted, 
economical  solution. 


-12- 


INPUT 


DSD  leads  to  conflicting  reports  to  management  -  -frequently  without 
qualification  as  to  the  source. 

The  integration  of  systems  (applications)  developed  in  the  DSD 
environment  lead  to  excessive  and  unanticipated  demands  being  made 
upon  host  mainframes. 

In  additi  on,  distributed  systems  development  can  actually  be 
counterproductive  even  while  demonstrating  surface  productivity 
improvement  by  getting  systems  developed  rapidly  and/or  on  schedule. 
This  coun terproduc t i v i ty  is  demonstrated  in  the  following  ways: 

Deterioration  of  data/ i nf ormat i on  quality. 

Unanticipated  expense  in  terms  of  operational  i nef f i c i en i es , 
maintenance  required,  etc.  which  results  in  substantially 
higher  life  cycle  costs. 

Unworkable  solutions  (systems  which  can  never  go  into 
production  because  of  poor  quality  or  excess  operational 
expense ) . 

Systems  which  continue  to  evolve  in  endless  cycles  of 
superficial  requirements  definition,  systems  analysis,  and 
quick  and  dirty  implementations  designed  to  show  quick 
"results"  without  ever  really  solving  the  problem. 

Among  the  tools  and  aids  recommended  by  INPUT  to  assure  quality  in  the 
DSD  environment  are  the  following: 
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Expanded  dictionaries,  directories,  encyclopedias,  and  glossaries  o-f 
data  and  information  are  required  to  assure  the  quality  o-f 
communication  among  systems  developers  and  users. 

A  meta  language(s)  is  necessary  to  describe  the  internal  inter-faces 
between  and  among  hardware/so-f tware/human  components  o-f  both  the 
development  and  target  computer/communications  networks.  (In  the 
study  report,  it  was  suggested  that  APL  might  serve  this  purpose, 
and  subsequent  research  tends  to  con-firm  the  potential  o-f  this 
1 anguage . ) 

Processing  and  data--flow  monitors  -for  predicting  and  controlling 
network  performance  are  required,  and  it  remains  INPUTs  opinion  that 
queuing  networks  provide  a  possible  -foundation  -for  such  monitors. 
(While  there  has  been  a  substantial  communications  problem  among 
operations  research  analysts,  computer  scientists,  and 
mathematicians  on  queuing  networks,  progress  is  currently  being 
made . ) 

An  integrated  document  storage  and  control  system  providing 
■features  -for  data  base  and  document  cert  i  f  i  cat  i  on  is  necessary. 
This  implies  not  only  data  base  integrity,  but  in-formation  -flow 
control  -for  purposes  o-f  security  and  protection,  and  inherent  in 
such  a  system  is  cer  t  i  f  i  cat  i  on  o-f  the  processing  algorithms  creating 
i  n-f  ormat  i  on  . 

There  is  also  a  need  -for  tools  to  analyse  productivity  tools.  The 
proliferation  o-f  "solutions"  to  the  productivity  problem  have  in 
themselves    become    part    o-f    the    productivity    problem    -    there  is 
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substantial  residual  cost  associated  with  many  approaches  to 
software  development  productivity  improvement.  Until  measurement  o-f 
productivity  in  the  systems  development  process  is  better 
understood,  evaluation  o-f  specific  tools  will  remain  difficult  - 
there-fore,  the  first  step  must  be  in  that  direction,  and  meaningful 
metrics  must  be  established. 

o  The  1985  Software  Quality  Assurance  study  was  directed  towards  major 
government  contractors  who  have  had  the  most  extensive  experience  in 
rigorous  quality  control  problems.  It  was  found  that  some  progress  has 
been  made  since  INPUT  identified  commitment  to  quality  as  the  foundation 
of  any  productivity  improvement  program  in  its  1980  report. 
Specifically,   it  was  found  that: 

The  companies  interviewed  had  established  quality  assurance  programs 
which  focused  attention  upon  quality  as  a  separate  and  important 
issue  over  the  system  life  cycle. 

The  individuals  responsible  for  the  operational  quality  assurance 
programs  seemed  knowledgeable  of  what  could  and  could  not  be 
accomplished,  and  were  dedicated  to  improving  the  quality  assurance 
programs  within  their  organizations. 

o  However,  there  seemed  to  be  little  concensus  among  the  various 
individuals  as  to  what  constitutes  an  effective  quality  assurance  program 
except  for  management  support  <broadbased  management  in  the  INPUT 
productivity  pyramid).  A  number  of  other  conclusions  were  reached  which 
support  this  lack  of  consensus. 
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The  diversity  of  tools  and  approaches  being  recommended  and  used 
leads  to  substantial  con-fusion  in  even  discussing  individual 
programs.     For  example: 

Reviews  can  be  extremely  detailed  or  quite  superficial,  and 
"walkthroughs"  can  range  -from  actual  code  inspections  to  simply 
signing  off  on  the  paperwork. 

Test  beds  established  by  the  quality  assurance  function  can 
perform  perfunctory  testing  as  part  of  the  product  shipment 
process,  or  can  actually  be  relied  upon  by  the  project  leaders 
to  help  them  in  debugging. 

There  are  no  magical  tools  for  any  phase  of  the  quality  assurance 
process  (including  code  inspection  and  testing).  As  one  executive 
stated:  "Lets  face  it,  if  any  of  us  (systems  houses)  could  get  a 
true  10'/.  software  productivity  advantage  no  one  else  could  compete. 
Therefore,  I  don't  believe  there  are  any  magic  bullets  out  there. 
And,  if  I  ever  find  one  you  can  be  sure  I'm  not  going  to  tell  anyone 
else  about  it!" 

Most  of  those  engaged  in  operational  quality  assurance  programs  feel 
that  the  technical  literature  on  productivity  and  quality  assurance 
is  the  work  of  theorists  who  do  not  understand  either  the  realities 
of  a  highly  charged  political  environment  (where  emphasis  is  on 
getting  the  product  out  the  door)  or  the  economics  of  software 
quality  assurance  (the  consensus  was  that  it  adds  approximately  10X 
to  the  development  costs).  Therefore,  the  theoretical  solutions  to 
some  of  the  problems  are  deemed  unacceptable  in  the  real  world. 


-16- 


INPUT 


There  was  one  point  on  which  there  was  consensus,  and  it  was 
negative  -  regardless  o-f  the  methodology  employed,  substantial 
amounts  o-f  documentation  are  generated.  This  adversely  a-f-fects 
acceptance  o-f  the  quality  assurance  -function,  and  also  impedes  the 
quality  assurance  organization  in  per-forming  its  work. 

There  was  a  general  -feeling  that  the  cost  of  quality  assurance  is 
difficult  to  justi-fy  because  the  results  cannot  be  measured.  This 
brings  us  right  back  to  the  problem  o-f  measurement  which  INPUT 
identified  as  the  need  -for  "tools  to  evaluate  tools".  This  is  also 
an  appropriate  lead  into  a  brief  analysis  of  artificial  intelligence 
and  expert  systems. 

o  In  the  United  States,  there  is  currently  considerable  enthusiasm  about 
the  potential  of  expert  systems  to  solve  many  categories  of  productivity 
problems  -  including  those  inherent  in  the  systems  development  process. 
INPUT  listed  ten  major  categories  of  expert  systems  and  some  potential 
personnel  impacts  in  Artificial  Intelligence  and  Expert  Systems  (1985). 
Exhibit  II. 3.  Some  of  the  conclusions  reached  in  that  study  as  they 
pertain  to  the  development,  testing,  operation  and  maintenance  of 
computer  systems  are  as  follows: 

The  tendency  in  the  computer  industry  has  always  been  to  under 
estimate  the  difficulties  of  software  development.  In  the  case  of 
expert  systems,  it  is  INPUTs  conclusion  that  the  building  of 
knowledge  bases  to  support  expert  systems  will  be  especially 
difficult  for  the  following  reasons: 


-17- 


INPUT 


CATEGORIES  OF  EXPERT 


EXHIBIT  11-3 
SYSTEMS  AND  REPRESENTATIVE 


IMPACTS 


CATEGORY 

SOME  POSSIBLE 
IMPACTED  PERSONNEL 

Interpretive  Systems 

Data  Entry  Personnel 
Business  Analysts 

Prediction  Systems 

Forecasters 

(Including  Consultants) 

Diagnosis  Systems 

• 

Doctors 

Field  Engineers 

Design  Systems 

Systems  and  VLSI  Designers 
Accounting  and  Financial 
Systems  Personnel 

Planning  Systems 

Programmers 
Schedulers 

Monitoring  Systems 

Project  and  Financial 
Control  Personnel 

Debugging  Systems 

Programmers 
Circuit  Designers 

Repair  Systems 

Field  Engineers 

Instruction  Systems 

Teachers 

Training  Personnel 

Control  Systems 

Industrial  Engineers 
Investment  Analysts 
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The  process  of  building  knowledge  bases  presents  a  level  of 
complexity  which  exceeds  that  o-f  normal  systems  development  and 
requires  highly  skilled  personnel  who  are  in  short  supply. 

There  is  every  indication  that,  except  in  very  narrow  domains, 
the  experts  are  either  unwilling  or  unable  to  de-fine  how  their 
knowledge  is  applied  in  complex  problem  solving  (which 
■frequently  is  highly  intuitive)  -  this  is  especially  true  in 
such  systems  problems  such  as  debugging. 

Because  o-f  the  very  nature  o-f  expert  systems,  the  potential  -for 
hidden  bugs  is  substantially  greater  than  in  conventional 
systems,  and  the  expert  may  -find  it  more  difficult  to  analyse 
the  "solution"  presented  by  the  expert  system  than  it  would 
have  been  to  solve  the  problem  -from  scratch. 

Having  performance  better  than  90%  of  human  "experts"  may  not 
be  acceptable,  when  problem  solutions  are  normally  obtained 
from  the  10'/.  who  specialize  in  solving  those  particular 
problems.     (For  example  in  medical  diagnosis.) 

The  question  of  human  differences  is  especially  important  in  the 
systems  development  process  where  is  has  long  been  recognized  that 
ranges  of  programmer/analyst  capability  vary  by  several  orders  of 
magnitude.  Barry  W.  Boehm  quotes  a  range  of  26  to  1  for  programmer 
productivity  in  the  debugging  process,  and  INPUTs  research  confirms 
a  range  of  at  least  that  magnitude  (in  fact,  there  are  those  who 
state  that  the  range  is  infinite  because  some  programmer/analysts 
would  never  be  able  to  solve  certain  problems). 
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It  is  INPUTS  opinion  that  the  classic  90-10%  rule  (which  states  90% 
of  a  system  is  normally  developed  in  10%  of  the  time  and  the 
remaining  10%  requires  90%  o-f  the  e-f-fort),  applies  to  individuals  as 
well  as  projects  -  in  other  words,  10%  o-f  the  programmer/analysts  do 
90%  o-f  the  productive  work,  and  the  remaining  90%  contribute  only 
10%  and  create  a  major  share  o-f  the  problems  -for  the  highly 
productive  10%  o-f  their  -fellow  workers.  It  is,  there-fore,  extremely 
dangerous  to  make  the  90%  more  productive  in  producing  systems  o-f 
marginal  quality  which,  in  all  probability,  will  require  either 
replacement  or  substantial  rework.  This  concern  about  expert 
systems  applies  as  well  to  other  applications  development  tools 
which  -facilitate  the  rapid  development  o-f  low  quality  systems  (that 
is  the  primary  reason  commitment  to  quality  is  at  the  base  o-f  the 
productivity  pyramid).  The  -fundamental  conclusion  remains  the  same 
-  it  is  counter-productive  to  develop  low  quality  systems. 

However,  it  is  not  our  intention  to  be  negative  about  either  expert 
systems  or  other  applications  development  tools,  and  INPUT  supports 
the  pro-found  observation  o-f  Ada  Augusta,  Countess  o-f  Lovelace,  about 
Charles  Babbage's  analytical  engine: 

"It  is  good  to  be  wary  o-f  exaggerating  the  ideas  that  arise 
■from  the  process  o-f  the  analytical  engine.  There  is  o-ften  a 
tendency,  in  considering  any  new  topic,  to  initially  overrate 
the  technology,  by  emphasizeing  the  interesting  or  amazing 
aspects  o-f  it.  Then,  when  we  realize  it  doesn't  meet  our 
expectations,  we  tend  to  underva 1  ue  the  true  condition  o-f  the 
technol ogy . 8 
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It  is  INPUTs  opinion  that  Lady  Lovelace  would  probably  be 
appalled  at  the  current  propensity  to  -first  overrate  and  then 
undervalue  applications  development  tools  -  including  the 
language  which  bears  her  name. 

o  All  past  INPUT  research  on  so-ftware  development  productivity  points  to 
the  -fact  that  there  are  no  easy  or  magical  solutions  to  the  so-ftware 
development  productivity  problem  and  the  research  -for  this  study  seems  to 
con-firm  that  conclusion. 
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III.  Project  Research 

A.  Ford  Aerospace  Western  Development  Laboratories 

o  Ford  Aerospace  was  interviewed  on-site  during  the  course  of  INPUTs  1980 
multi-client  study  and  was  selected  as  the  primary  research  source  -for 
this  study  -for  the  following  reasons: 

The  productivity  improvement  e-f-forts  o-f  Ford  Aerospace  were 
spec  i  i  i  cal  1  y  directed  towards  software  development  o-f  embedded, 
on-line,  real-time  computer  systems.  INPUT  has  concluded  that 
these  are  o-f  particular  interest  to  Toshiba,  and  we  agree  that  the 
complexity  o-f  todays  computer/communications  networks  (even  those  o-f 
a  commercial  nature)  present  all  computere  systems  vendors  and  users 
with  development  problems  which  are  comparable  to  those  which  have 
been  traditionally  -faced  by  the  aerospace  industry. 

The  original  research  indicated  clearly  what  INPUT  considers  to  be 
the  complexity  o-f  evaluating  the  effectiveness  of  using  software 
development  tools. 

Representatives  of  Ford  Aerospace  have  been  quite  open  in  reporting 
the  results  of  their  productivity  improvement  efforts,  and  this  has 
not  been  true  of  most  comparable  efforts. 

Ford  Aerospace  western  Development  Laboratories  is  conveniently 
located  to  INPUTs  corporate  offices. 

o  At  the  time  of  INPUTs  earlier  research,  INPUT  found  that  Ford  Aerospace 
was  mounting  a  substantial  effort  to  improve  software  development 
productivity  through  both  environmental  considerations  (improved  office 
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workspace  and  availability  o-f  terminals),  and  through  the  adoption  and 
extension  o-f  the  Programmer  Workbench  <PUB) .  (It  was  reported  to  INPUT 
that  more  than  $1  million  had  been  spent  enhancing  the  PUIB.)  In  1980,  a 
coordinated  set  of  methods,  standards  and  procedures  were  being  proposed 
■for  all  -future  projects,  and  there  was  quite  a  bit  o-f  enthusiasm  at  the 
executive  level  about  the  progress  which  was  being  made. 

o  As  mentioned  earlier  INPUTs  approach  was  to  interview  at  various  levels 
within  the  company,  and  as  we  proceeded  down  -from  the  executive  level,  we 
encountered  the  -following  reactions  to  the  proposed  corporate  standards 
■for  the  development  environment. 

The  manager  o-f  one  major  project  was  -familiar  with  the 
standardization  e-f-fort,  but  stated  that  his  major  strength  was  the 
ability  to  keep  together  a  compatible  team  -from  one  project  to  the 
next,  and  this  was  accomplished  because  his  employees  liked  his 
management  style  and  his  project  control  system.  His  attitude  was 
that  any  attempt  to  -force  a  change  would  increase  both  costs  and 
turnover  on  his  project,  and  he  did  not  think  anyone  would  want  to 
take  responsibility  -for  that.  His  support  o-f  the  corporate  e-f-fort 
was  limited  to  new  projects  where  new  teams  are  being  brought 
together  -  not  new  projects  being  given  to  an  established  work  unit 
<such  as  his). 

A  project  leader  interviewed  stated  she  had  heard  about  the  PWB  and 
associated  standards  but  wasn't  "worried"  because  "my  project  has  2 
or  3  years  to  run,  and  by  that  time  everyone  will  have  -forgotten 
about    the    standards".      Her    general    attitude   was    that    she  would 
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protect  her  projects  -from  unnecessary  Intrusion  -from  a  bunch  o-f 
"corporate  types" . 

A  programmer  interviewed  merely  stated  that  it  sounded  like  a  lot  o-f 
"paperwork"  to  him,  but  he  would  depend  upon  his  immediate  manager 
to  take  care  o-f  the  problem. 

While  this  highly  individualized  and  resistant  attitude  may  seem 
quite  -foreign  to  Japanese  management,  it  is  especially  common  in 
so-ftware  development  in  the  United  States.  There  is  a  general 
tendency  to  permit  managers  and  project  leaders  who  are  perceived  as 
being  highly  productive  to  continue  doing  what  they  are  doing  as 
long  as  they  meet  rather  poorly  de-fined  objectives.  And,  in  all 
truth,  there  is  some  concrete  evidence  that  introduction  o-f  changes 
in  methodology  or  tools  will  adversely  impact  productivity  o-f  those 
accustomed  to  a  di-f-ferent  working  environment. 

o  In  1984,  the  -former  so-ftware  engineering  director  o-f  Ford  Aerospace 
Western  Development  Laboratories  published  a  care-fully  thoughtout  article 
on  the  results  o-f  the  -five  year  e-f-fort  to  improve  so-ftware  productivity. 
The  -first  and  most  discouraging  -fact  reported  was  that  the  cost  per 
source  instruction  (adjusted  -for  inflation)  had  gone  up  between  197?  and 
1983  (the  period  covered  by  the  all  out  e-f-fort).  However,  there  were 
some  reported  changes  to  explain  and  o-f-fset  this  seeming  adverse  impact 
on  productivity. 

First  o-f  all,  it  was  pointed  out  that  the  systems  being  developed  in 
1983  much  bigger  and  more  complex  with  much  greater  "documentation 
and  record-keeping  demands". 
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Design  level  work  in  Program  Design  Language  <PDL)  "-far  exceeded  the 
kind  o-f  detail  thought  necessary  in  1978-'79>n. 

Unique  devices  were  attached  to  the  computers  and  they  required 
special  software  and  timing  analysis. 

The  burden  placed  on  the  system  at  the  man-machine  inter-face  had 
greatly  increased. 

Each  computer  performed  many  interleaved  -functions  and  the  internal 
housekeeping  -function  lowered  e-f-fective  throughput  which,  in  turn, 
required  -further  redesign  and  reprogr  amm  i  ng . 

And,  it  was  then  pointed  out  that  quality  (in  terms  o-f  detected 
discrepancies  during  final  integration)  had  been  improved,  and  both 
estimating  and  cost  control  had  been  improved.  The  quality  o-f 
documentation  was  "so  high  that  it  wa  probably  excessive." 

The  general  conclusion  reached  was  that  quality  and  consistency  had 
been  achieved  at  the  expense  o-f  productivity.  The  report  went  on  to 
analyze  the  continuing  "productivity  problem0  in  more  detail  and 
this  will  be  discussed  under  conclusions  and  recommendations. 

Discreet  inquiries  recently  revealed  that  at  least  some  projects  at  Ford 
Aerospace  remain  outside  the  standard  development  procedures  which  were 
being  established  in  1980,  and  some  project  leaders  definitely  feel  the 
paperwork  involved  in  using  many  of  the  new  tools  adversely  affects  their 
groups  productivity.  The  general  attitude  seems  to  be  that  the 
requirements  for  tools  varies  from  project  to  project  and  selection 
should  be  left  to  the  project  leaders. 
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o  Based  on  the  reported  results  at  Ford  Aerospace,  it  is  also  possible  to 
raise  other  questions  concerning  the  true  impact  o-f  the  productivity 
improvement  e-f-fort  which  could  not  be  pursued  with  in  the  limits  o-f  this 
study. 

Since  it  is  INPUTs  experience  that  the  env i oronmental  changes  being 
pursued  by  Ford  Aerospace  (o-f-fice  space,  terminal  access,  etc.) 
normally  result  in  productivity  improvement,  was  the  impact  o-f  the 
standards,  methods,  and  tools  <PU)B,  PDL,  etc.)  actually  more 
negative  than  the  reported  modest  increase  in  the  cost  per  source 
code  instruction? 

Were  all  o-f  the  costs  o-f  the  productivity  improvement  program 
included  in  the  costs  being  measured? 

Did  the  tools  employed  contribute  to  the  increase  in  complexity 
which  was  observed  over  the  5  year  period? 

Were  the  projects  which  did  not  employ  the  standard  tools  and 
methodologies  included  in  the  measurements?  And,  what  was  their 
realtive  impact  on  the  cost  per  source  code  instruction? 

Were  Ford  Aerospace  projects  such  as  the  York  an t  i  -a i  rcra-f  t  gun  (a 
$1.8  billion  project  which  was  scrapped  in  1985)  included  in  the 
cost  per  source  code  instruction  calculations?  (It  became  obvious 
■from  the  published  accounts  that  some  o-f  the  embedded  systems 
mal i unct i oned  rather  dramatically,  and  this  calls  into  question  any 
statements  about   improved  quality.) 
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o  INPUT  believes  the  Ford  Aerospace  experience  is  especially  important  in 
demonstrating  that,  despite  sincere  efforts  in  establishing  a 
comprehensive  software  development  productivity  improvement  program,  the 
problems  of  productivity  improvement  seem  to  remain.  And,  Ford  Aerospace 
is  not  optimistic  about  the  prospects  -for  improvement  because  they  have 
reached  other  conclusions  based  on  their  experience: 

"Integration  and  testing,  although  a  large  part  o-f  the  total  cost, 
is  not  a  very  promising  area  -for  productivity  improvement.  It  is 
near  the  end  o-f  the  development  cycle  and  consequently  has  little 
■flexibility  and  very  little  multiplier  effect.  ...  Clever 
simulators,  automatic  test  sequencing  and  recording,  and  even 
partial  automatic  test  plan  generation  would  not  cause  large  savings 
in  this  part  o-f  the  development  cycle." 

And,  they  con-firm  what  we  all  know  about  coding  with  the  -following 
statement:  "Suppose  we  could  go  to  machine  instructions  directly 
■from  a  design  language  like  PDL,  all  on  the  computer.  That  entire 
process  now  represents  only  15%  or  less  o-f  the  cost  of  producing  the 
kind  o-f  software  we  are  concerned  with,  and  a  good  portion  of  that 
is  occupied  by  module  and  unit  testing.  It  appears  we  would  be  very 
lucky  indeed  to  realize  an  overall  improvement  in  productivity  of 
10%  or  more  in  this  area.  The  technology  to  go  to  machine 
instructions  directly  from  a  design  language  is  still  a  long  way 
off.  Such  a  large  R&D  investment  for  the  potential  productivity 
yield  does  not  seem  cost  effective." 

B.  Hewl ett-Packard 
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It  was  pointed  out  earlier,  that  most  computer  and  systems  companies  are 
not  open  about  the  so-ftware  development  productivity  problem.  This  is 
true  -for  two  reasons: 

The  problem  seems  to  remain  regardless  o-f  how  much  e-f-fort  is 
expended  in  pursuit  o-f  its  solution. 

The  people  attempting  to  solve  the  productivity  problem  become 
personally  involved  and  are  reluctant  to  publicize  results  which 
expose  the  persistant  nature  o-f  problems  which  they  are  supposed  to 
be  able  to  solve.  <In  addition,  i-f  true  progress  were  made  it  is 
probable  that  the  particular  solutions  would  be  considered  to  give  a 
competitive  advantage  and  be  kept  proprietary.) 

The  net  result  is  that  whether  there  is  success  or  -failure  the 
tendency  is  keep  internal   tools  and  aids  proprietary. 

In  order  to  obtain  some  insight  into  what  is  actually  happening  at 
Hewlett-Packard,  a  con-f  i  den  t  i  al  interview  was  conducted  with  Gopal  Kapur 
o-f  Kapur  and  Associates  who  specializes  in  so-ftware  productivity 
improvement  consulting.  Mr.  Kapur  has  conducted  seminars  at  HP,  and 
Kapur  and  Associates  has  developed  a  li-fe  cycle  and  project  management 
system  (Kapur  Method/2000)  which  is  being  used  by  some  o-f  their  clients, 
and  which  is  currently  beginning  to  be  marketed  commer i c i al 1 y . 

Mr.  Kapur  gave  us  the  -following  in-formation  concerning  HP  productivity 
improvement  e-f-forts. 

HP  has  a  proprietary  development  environment  called  HP  Li-fe  Cycle 
which    is    a    li-fe    cycle    and    project    management    methodology,  and 
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includes  an  integrated  work  bench.  There  are  two  versions  o-f  HP 
Life  Cycle,  one  for  engineering  and  one  -for  internal  HP  business 
app 1 i  cat  i  ons . 

Mr.  Kapur  stated  that  it  is  not  HPs  management  style  to  en-force  use 
o-f  these  tools,  or  to  impose  what  he  views  as  necessary  discipline 
in  the  systems  development  process. 

He  has  observed  a  general  reluctance  to  talk  about  the  actual 
experience  with  HP  Li-fe  Cycle,  but  his  experience  with  speci-fic 
projects  (business  applications)  within  HP  would  seem  to  demonstrate 
that  significant  progress  in  such  areas  as  specification 
preparation,  structured  walkthroughs,  and  test  beds  has  not  been 
made . 

He  stated  that  both  software  engineering  and  data  processing 
management  regularly  attend  his  seminars,  and  it  is  obvious  that 
they  recognize  the  need  for  improvement  in  the  application  of  HP 
Life  Cycle  if  not   its  technical  and  functional  content. 

o  It  is  INPUTs  opinion  that  HP  is  probably  representative  of  the  larger 
computer  manufacturers  -  they  have  provided  tools,  but  except  for 
documentation,  individual  projects  are  permitted  a  great  deal  of 
flexibility  in  whether,  or  how,  such  tools  are  employed.  Kapur  concurs 
in  INPUTs  assessment  that  particular  tools  are  not  really  so  important  as 
how  they  are  applied,  and  the  general  physical  and  management  environment 
in  which  they  are  applied.  (For  example,  Kapur  and  Associates  has 
adopted  a  policy  of  only  answering  telephone  calls  between  1:00  and  4:00 
PM  and  feels  this  improves  their  own  internal  productivity  as  much  as 
employing  any  specific  methodology.) 
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C.  Tektronix.  Memorex  Communications,  CXI 

o  In  order  to  get  a  general  -feel  -for  how  smaller  companies  approach 
productivity  improvement,  INPUT  conducted  a  confidential  interview  with 
Warren  Davidson  who  has  served  as  Project  Manager  with  Tecktronix, 
Manager  o-f  Technical  Services  with  Memorex  Communications  Group,  and  is 
currently  Manager  o-f  So-ftware  Services  with  CXI, Inc.  (Micro-main-frame 
products).  In  all  o-f  his  positions,  Mr.  Davidson  has  been  responsible 
■for  control  and  release  o-f  so-ftware  and  per-formance  measurement,  and  has 
been  instrumental  in  attempting  to  establish  Project  Management  and 
Control  Systems. 

o  Mr.  Davidson's  general  analysis  is  that  so-ftware  development  teams 
normally  pre-fer  to  develop  their  own  development  tools  (or  tailor  others 
to  suit  their  particular  purposes),  resist  -formal  project  management  and 
control  systems  (or  consider  it  an  overhead  exercise  and  do  the  minimum 
they  can  to  comply),  and  work  in  collusion  with  marketing  (and  customers) 
to  -force  products  through  the  release  process. 

o  It  is  his  impression  that  systems  software  development  (embedded 
communications  systems,  operating  systems,  etc.)  is  much  more  difficult 
to  control  than  commercial  applications  development  because  systems 
programmers  and  engineers  do  not  respect  anyone  without  their  detailed 
technical  knowledge  of  their  systems  (and  this  extends  to  management  as 
it  gets  removed  from  the  day-to-day  technical  problems).  It  is  difficult 
for  anyone  (quality  control  or  line  management)  to  impose  project  control 
on  implementors  of  complex  systems  because  the  project  leaders  always 
fall  back  on  the  question  of  who  is  going  to  be  responsible  for  the 
schedule.     (Mr.  Davidson  stated:   "the  time  when  you  really  have   to  watch 
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out  is  when  they  agree  to  run  their  projects  by  your  rules  -  then  you 
Know  they  are  going  to  blame  any  problems  on  you.) 

o  The  result  has  been  that  effective  project  management  and  control  systems 
are  extremely  difficult  to  install  -  for  example,  one  does  not  currently 
exist  at  CXI  and  there  is  little  hope  of  getting  one  installed  despite 
the  fact  that  the  President  of  the  company  specifically  hired  Mr. 
Davidson  to  do  just  that.  As  he  so  aptly  puts  it:  "When  you  don't  know 
what  is  going  on,  you  obviously  can't  measure  it;  if  you  can't  measure  it 
you  can't  control  it;  and  if  you  can't  control  it,  it  is  ridiculous  to 
talk  about  improving  productivity.    We  have  problems." 

o  In  his  previous  assignments,  Mr.  Davidson  had  succeeded  in  installing  an 
"automated"  software  testing  system  (basically  a  custom  VTAM  simulator) 
which  feed  data  into  a  project  control  and  product  performance  evaluation 
system.  However,  the  simulator  was  developed  by  an  outside  firm  and  the 
reporting  systems  were  developed  in  house  and  were  considered 
propr  i  etary . 

D.  Dialogic  Systems.  Inc. 

o  Dialogic  has  developed  a  Development  Center  WorkBench  which  is  based  on 
IBMs  Interactive  System  Productivity  Facility  <ISPF)  and  is  designed  to 
off-load  COBOL  development  activities  from  IBM  mainframes.  The  cost  of 
less  than  $3,000  per  user  is  a  substantial  reduction  from  the  costs 
associated  with  interactive  development  using  IBM  mainframes,  and 
improved  development  tools  are  reported  to  provide  over  a  20'/. 
productivity  improvement  compared  to  those  on  IBM  mainframes.  The 
Development    Center    WorkBench    can    be    used    either     locally  (directly 
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connected  to  the  Block  Multiplexor  channel  or  connected  remotely  over 
multiple  communication  lines. 

However,  INPUTs  interest  was  in  determining  how  the  Development  Center 
WorkBench  itsel-f  was  developed  -for  the  DIALOGIC/10  MidFrame  computer 
which  consists  o-f  multiple  -functional  processors  (up  to  31  Motorola  68000 
microprocessors)  operating  in  parallel.  These  high  performance  embedded 
microprocessors  are  as  -follows: 

Host  communications  processor. 

Remote  terminal  processor. 

Local   terminal  processor. 

Diagnostic  processor. 

Multiple  application  processors. 

Multiple  -file  processors. 

It  is  our  understanding,  based  upon  a  telephone  interview,  that  the 
original  development  work  -for  the  Development  Center  WorkBench 
(approximately  250  man  years  o-f  e-f-fort)  was  done  in  C  Basic  and  Pascal  on 
a  VAX  supermini  under  UNIX,  bu  t  the  enhancement  and  maintenance 
activities  (which  have  been  estimated  to  require  another  250  man  years) 
are  being  done  on  the  WorkBench  itseW.  I-f,  in  -fact,  this  bootstrapping 
has  occurred  it  would  indicate  language  and  target  processor  capability 
which  could  potentially  be  very  attractive  -for  developing  high 
per-formance  applications  in  an  SNA  network  environment.  Unfortunately, 
the  scope  of  this  study  did  not  permit  us  to  substantiate:  1)  whether  the 
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Development  Center  WorkBench  is  really  being  used  -for  its  own  enhancement 
and  maintenance,  2)  exactly  what  the  $3,000  per  user  cost  -figure 
contains,  or  3)  whether  there  is  any  plan  for  marketing  another  version 
of  the  WorkBench  specifically  aimed  at  distributed  architectures. 

E.  Other  Interviews 

o  As  mentioned  earlier  in  this  report,  it  was  decided  to  review  brie-fly  the 
status  of  So-ftware  Engineering  Economics  (as  presented  by  Barry  W.  Boehm 
in  his  book  on  the  subject),  In-formation  Engineering  <as  originally 
presented  by  James  Martin  and  Clive  F i nk 1 este i n )  ,  and  Queuing  Networks 
(as  pursued  by  Dr.  Ralph  L.  Disney).  The  reasons  -for  INPUTs  interest  in 
these  three  areas  will  become  apparent  when  our  conclusions  and 
recommendat  i  ons  are  presented.  Brie-fly  stated,  the  current  status  of  the 
three  areas  are  as  follows: 

While  INPUT  still  feels  that  Mr.  Boehm's  book  has  made  a  substantial 
contribution  to  the  metrics  of  software  development,  and  could  serve 
as  the  basis  for  workable  models  for  project  estimating, 
productivity  (performance)  measurement,  and  project  control;  there 
are  some  problems  with  the  practical  application  of  much  of  his 
knowl edge . 

Mr.  Boehm  as  Director  of  Software  Research  and  Technology  at 
TRW,  Inc.  is  concerned  primarily  with  government  contracts  in 
aerospace  and  defense.  Detailed  analysis  of  the  specific 
success  or  failure  is  difficult  because  the  specific  tools  are 
considered  proprietary,  and  the  projects  are  classified. 
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A  con-f  i  dent  i  al  interview  with  the  TRW  credit  operation 
disclosed  that  Mr.  Boehrn's  work  was  economically  justified  only 
on  government  projects  with  stringent  quality  control 
requirements.  (TRW  estimates  that  quality  assurance  using  SDM 
70  adds  15X  to  development  costs  on  commercial  projects  even 
a-fter  the  documentation  required  is  reduced  substantially.) 

Mr.  Martin  and  Mr.  Finklestein  came  to  a  parting  of  the  ways  shortly 
a-fter  writing  their  book  on  Information  Engineering.  The  problem 
seemed  to  revolve  around  the  practical  application  of  Information 
Engineering  and  what  the  term  really  meant.  Since  then  Mr.  Martin 
has  become  associated  with  DDI,  and  it  appears  that  the  product  is 
being  used  to  define  the  concept.  One  thing  becomes  clear  - 
Information  Engineering  is  not  a  science,  and  the  term  has  even  less 
meaning  than  Fourth  Generation  Languages  (another  ill-defined  term 
from  the  fertile  mind  of  Mr.  Martin). 

Dr.  Disney  has  specialized  in  Queuing  Networks  for  over  25  years, 
and  gradually  this  Operations  Research  tool  has  become  accepted  by 
computer  scientists  as  being  of  considerable  assistance  in  complex 
computer  scheduling  and  resource  allocation  problems.  Essentially, 
much  to  the  consternation  of  some  mathematicians,  Queuing  Networks 
have  been  remarkably  accurate  in  predicting  device  utilizations  and 
throughputs  (errors  seldom  exceed  5'0  ;  and,  while  less  reliable  as 
predictors  of  queue  length  and  waiting  time,  error  rates  usually 
less  than  25X.  The  author  of  this  study  has  consulted  with  Dr. 
Disney  over  the  years,  and  there  are  particular  reasons  to  be 
concerned  about  Queuing  Networks  at  this  time: 
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Hierarchical  computer/communications  networks  with  systems 
<and  data)  distributed  over  main-frames,  minicomputers,  and 
microprocessors  present  resource  allocation  problems  which  are 
substantially  more  complex  than  any  encountered  by  operating 
systems  on  central  processors. 

It  has  long  been  the  authors  intuitive  belie-f  that  the 
organization  of  large  so-ftware  development  projects  and  the 
attendant  communications  <data  and  information  -flow)  problems 
are  amenable  to  Queuing  Network  modeling;  and,  this  intuitive 
■feeling  even  stronger  as  project  teams  go  on-line  with  advanced 
software  development  tools. 

Dr.  Disney  agrees  that  Queuing  Networks  are  promising  -for  the 
more  complex  distributed  systems  which  are  currently  being 
developed;  he  stated:  "everyone  seems  to  use  the  term  'robust' 
these  days,  and  I  guess  Queuing  Networks  are  robust." 

He  was  not  prepared  to  comment  about  software  development  work 
units,  but  he  was  sufficiently  interested  to  schedule  an  April 
meeting  with  us  in  Palo  Alto. 

INPUT  has  also  stated  that  APL  might  provide  the  necessary  me ta-1 anguage 
for  concise  communication  across  systems  interfaces.  Ule  conducted  a 
confidential  interview  with  an  IBM  employee  who  has  been  associated  with 
APL  since  its  beginning  in  the  IBM  Research  Center  in  Yorktown  Heights. 
Not  surprisingly,  he  agrees  with  our  assessment  of  the  potential  of  APL, 
and  he  stated  that  recent  releases  have  provided  enhanced  capability 
which    would    facilitate    reliable    "foreign    language"    execution  across 
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systems  inter-faces.  However,  the  battle  within  IBM  concerning  APL 
continues  unabated  a-fter  more  than  20  years. 
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IU.  Conclusions  and  Recommendations 
A.  Conclusions 

o       The  speci-fic  conclusions  reached  -from  this  study  are  as  -follows: 

There  are  no  general  all  encompassing  solutions  to  the  productivity 
improvement  problem  -  no  one  tool  or  set  o-f  tools  is  available  which 
are  either  acceptable  o-f  workable  in  all  situations. 

Major  computer  manu-f ac turers  and  systems  developers  are  inclined  to 
develop  their  own  set  o-f  tools  <or  heavily  modi-fy  those  which  are 
commercially  available),  and  attempt  to  standardize  on  the  use  o-f 
those  tools  -for  purposes  o-f  project  management  and  quality  control. 
There  is  little  reason  to  believe  that  these  e-f-forts  have  been 
e-f-fective  in  either  producing  clearly  demonstratabl  e  productivity 
improvement  or  in  standardizing  the  way  so-ftware  is  developed  or 
managed . 

Smaller  companies  (with  smaller  work  units)  are  especially  resistant 
to  the  imposition  o-f  methodologies  which  address  the  li-fe  cycle 
because  o-f  increased  paperwork.  There  is  still  the  classic  tendency 
to  rush  into  programming  with  inadequate  requirements, 
spec  i -f  i  cat  i  ons  ,  analysis  and  design;  and  programmers  continue  to 
resist  documentation.  There  is  also  a  general  -feeling  that  highly 
productive  individuals  and  work  units  should  not  be  disturbed  -for 
■fear  that  productivity  will  be  adversely  impacted  (which  may  be 
true)  . 

Generally  speaking,  there  seems  to  be  concensus  that  tools  should 
address   the   li-fe   cycle   and    that    some    type   o-f    integrated  software 
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development  work  bench  with  ready  access  to  centralized,  on-line 
documentation  is  required.  (Massive  volumes  of  paper  documentation 
is  a  sure  sign  o-f  a  project  which  is  in  trouble.) 

It  is  generally  -felt  that  systems  complexity  is  increasing  more 
rapidly  than  any  efforts  to  improve  productivity,  but  this 
especially  difficult  to  quantify  because  there  are  no  accepted 
metrics  or  agreement  on  what  contitutes  software  productivity. 

Despite  changes  in  terminology  (such  as  software  engineering), 
software  development  remains  a  relatively  undisciplined  art  rather 
than  a  science.  Twenty  years  ago,  computer  architects  (such  as  Dr. 
Fred  Brooks  and  Dr.  Gene  Amdahl)  felt  that  engineering  discipline 
could  be  applied  to  the  software  development  process  -  to  date, 
there  has  been  very  little  progress  in  doing  this. 

Past  experience  has  shown  that  success  or  failure  of  software 
projects  seems  to  depend  more  on  "good  management"  rather  than  the 
application  of  particular  tools,  methodologies,  or  formal  approaches 
to  productivity  improvement.  Unfortunately,  there  does  not  seem  to 
be  any  clear  indication  of  what  consitutes  "good  management"  of 
software  development.  It  is  possible  to  isolate  vastly  different 
management  "styles"  which  seem  to  have  contributed  to  both 
spectacular  successes  and  failures  in  software  projects. 

o       INPUT  is  convinced  that   the  productivity  pyramid  (Exhibit  II. 1)  forms  a 
general    basis    for    the    management    of    software    projects    and    for  the 
improvement   of    productivity.      We    are    also  convinced    that  productivity 
must  be  viewed  at  four  performance  levels: 
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Hardware-software 
Human-machine  dyad 
Work  un  i  t 
Institutional 

o       When    viewed    in    this    manner,    several     additional     conclusions    can  be 
reached : 

There  is  tendency  to  believe  that  hardware  costs  are  decreasing  so 
rapidly  that  sloppy  software  can  be  tolerated  and  overcome  by 
processing  power.  INPUT  believes  that  commitment  to  quality  extends 
to  hardware-so-f  tware  performance  and  that  "quick  and  dirty" 
solutions  (including  tools  which  create  hardware  performance 
problems)  are  counter  productive. 

Most  attention  at  the  human-machine  dyad  has  been  addressed  to 
making  the  human  more  productive.  In  the  United  States  this  has 
taken  the  form  assuming  that  English  is  an  appropriate  human-machine 
interface.  INPUT  does  not  agree  with  this  assumption  and  feels  that 
unrestrained  endorsement  of  English  language  tools  will  be  counter 
productive.  E.  W.  Dijstra  has  expressed  the  feeling  that  English 
tends  to  obscure  the  very  power  of  the  computer,  and  our  current 
research  on  APL  elicited  the  same  reaction  -  "Everyone  complains 
about  all  of  the  symbols  in  APL,  but  there  are  many  problem 
solutions  which  can't  be  descibed  in  English."  In  addition,  human 
capabilities  vary  so  much  that  no  single  man-machine  interface  is 
best  in  all  cases. 
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At  the  work  unit,  there  has  long  been  a  general  awareness  that 
adding  more  people  does  not  normally  improve  schedules,  and  large 
work  units  are  not  as  productive  as  small  work  units.  Despite  this 
awareness,  there  has  been  a  general  tendency  to  add  personnel  to 
both  the  development  work  units,  and  "in  support"  of  the  work  unit. 
There  is  every  indication  adding  more  people  and  making  those  o-f 
lower  ability  "more  productive"  tends  to  lower  work  unit  performance 
substantially.  During  the  course  o-f  research  -for  this  study,  Gopal 
Kapur  stated  he  has  a  guaranteed  way  o-f  improving  software 
productivity  at  least  507.  -  he  challenges  his  clients  to  remove  the 
least  productive  50'/.  o-f  their  employees,  and  he  agrees  that  the  last 
thing  which  should  be  done  is  to  improve  the  productivity  o-f  poor 
performers  (because  their  work  is  really  counter-productive  to  work 
un  i  t  performance) . 

Institutional  performance  can  only  be  improved  by  having  end  user 
involvement  and  broadbased  management.  There  is  a  general  tendency 
for  various  work  units  to  shift  responsibility  to  others.  For 
example,  computer  architects  have  been  inclined  to  leave  certain 
problems  to  software  in  order  to  keep  costs  down.  A  classic  case  of 
this  was  the  decision  not  to  provide  indirect  addressing  in  the  IBM 
System/360  architecture  -  there  is  no  question  that  this  was  counter 
productive  from  the  point  of  view  of  both  the  corporation  and  its 
customers.  (The  current  trend  towards  RISC  machines  is  especially 
susceptible  to  these  types  of  counter-productive  trade-offs,  and 
must  be  carefully  managed  with  institutional  performance  in  mind.) 
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The  analysis  of  the  Ford  Aerospace  experience  with  productivity 
inmprovement  provided  the  -following  list  o-f  reasons  that  so-ftware 
development  is  so  di-f-ficult: 

Incomplete  or  inaccurate  software  requ i remen ts . < 1 ) 

Incomplete  or  inaccurate  inter-face  de-f  i  n  i  t  i  on  .  ( 2) 

Trying  to  -fit  too  much  in  a  given  memory  space. (3) 

Trying  to  do  too  much  computing  in  too  little  processing  time. (4) 

Trying  to  do  too  many  things  simultaneously  inside  a  single 
compu  ter . <  5) 

Internal  con-flict  -for  resources.  (6) 
Queuing. <7) 

Too  many  types  and/or  numbers  o-f  inter-faces  with  outside 
hardware . (8) 

Error  detection  and  correct i on . (9) 

Documentat ion . (10) 

This  list  can  then  be  classified  as  follows: 

Items  1  and  2  are  familiar  project  management  and  systems 
engineering  problems  which  have  been  discussed  in  terms  of  the 
product  i  v  i  ty  pyrarni  d . 

Items  3  through  7  are  concerned  with  software  system  architecture 
and    all    represent    parts    of    the    same    problem    -    todays  software 
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architectures  were  -fundamentally  designed  -for  another  technological 
era.  Perhaps  no  where  is  this  more  evident  than  in  IBMs  Systems 
Network  Architecture  with  its  heavy  emphasis  upon  large  host 
main-frames  with  multiple  levels  o-f  operating  systems  overhead.  It 
is  -felt  that  advances  in  microprocessor  technology  and  economics 
make  this  problem  amenable  to  solution.  (Fundamentally,  this 
demonstrates  the  importance  o-f  the  hardware-so-f  tware  performance 
level  -  housekeeping  and  overhead  consume  so  much  o-f  the  systems 
resources  that  they  are  not  applied  to  solving  the  problem  at  hand.) 

Item  8  can  clearly  be  demonstrated  at  all  levels  in  the  processing 
hierarchy,  but  the  problem  is  particularly  acute  in  o-f-fice 
automation  (local  area  networks)  where  diverse  devices  must  be 
accommodated  and  new  technology,  such  as  optical  memories,  must 
inter-face  with  obsolete  systems  software  (both  operating  systems  and 
data  base  management  systems).  Once  again  the  problems  listed  in  3 
through  7  become  apparent  and  there  is  great  risk  that  performance 
at  the  hardware-software  level  will  result  in  the  development  of 
unworkable  systems  where  response  times  become  unacceptable. 
Standards  are  a  possible  solution  in  this  area,  but  IBMs  dominance 
of  the  US  computer  industry  permits  the  establishment  of  de  facto 
standards  and  virtual  control  over  most  industry  standards  activity. 
(For  example,  it  is  impossible  to  establish  standards  for  optical 
storage  devices  before  IBM  embraces  the  technology.) 

The  complexity  of  shared  software  architectures  (in  the  current 
environment,  the  VM/MVS/UNIX  strategy  of  IBM,  under  the  SNA 
architecture,    establishes    levels   of    complexity  which    threaten  to 
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extend  down  to  the  microprocessor  level)  make  error  detection  and 
correction  (Item  9)  extremely  di-f-ficult  and  this  in  turn  has  obvious 
quality  and  productivity  impacts.  Testing  and  integration  in  an 
environment  dictated  by  IBM  will  become  increasingly  di-f-ficult,  and 
the  general  tendency  to  be  IBM  compatible  will  turn  out  to  be  a 
vicious  trap. 

Ford  Aerospace  describes  documentation  (Item  10)  as  a  "massive, 
costly,  and  time  -consuming  element  of  so-ftware  development"  which 
is  presently  overdone  and  "is  used  mostly  to  convince  semiskilled 
project  managers  and  customers  that  the  so-ftware  is  going  together 
properly  and  addressing  all   the  system  requirements." 

o  INPUT  concludes  that  much  of  our  past  analysis  o-f  the  productivity 
problem  is  being  con-firmed  -for  all  types  o-f  systems  development  projects. 
It  is  an  extremely  complex  problem  which  will  not  be  solved  by  any 
magical  tool  which  places  the  emphasis  on  generating  code  and  getting 
■fast  results.  As  the  environment  becomes  more  complex  there  is  no  less 
need  -for  tools  to  -facilitate  program  development  and  more  need  -for  tools 
to  analyse  data  and  in-formation  -flow. 

o  The  -final  conclusion  is  that  the  scope  o-f  this  study  did  not  permit 
su-f-ficient  analysis  o-f  any  specific  productivity  tools  being  employed  by 
computer  manufacturers  or  those  available  from  commercial  vendors. 
However,  it  is  our  opinion  that  most  will  work  if  employed  after  assuring 
the  base  of  the  productivity  pyramid  is  in  place,  but  none  will  improve 
product i v i ty  wi thout  that  foundation. 

B.  Recommendations 
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It  is  not  recommended  that  Toshiba  attempt  to  adapt  the  productivity 
programs  and/or  tools  o-f  other  vendors  to  its  own  use.  There  is  no 
indication  that  any  o-f  these  tools  have  achieved  dramatic  improvements  in 
productivity.  The  problem  is  too  complex  for  simple  solution  and  undue 
emphasis  upon  tools  can  be  a  distraction  -for  establishing  a  comprehensive 
productivity  improvement  program. 

However,  we  do  recommend  that  Programmer  Work  Benchs  and  the  latest 
developments  in  Life  Cycle  and  Project  Management  Methodologies  be 
explored  i  -f  you  have  not  already  done  so.  While  INPUT  does  not  endorse 
any  products,  both  the  Kapur  Method/2000  and  the  Dialogic  Development 
Center  WorkBench  were  both  o-f  sufficient  interest  to  warrant  additional 
analysis,  and  Maestro  (developed  by  Softlab  GmbH  and  implemented  on 
Four-Phase  Systems  hardware)  has  provided  a  general  programming 
environment  which  has  been  been  commercially  available  for  sometime. 
Addresses  for  marketing  outlets  here  in  the  United  States  are  provided  in 
Append  i  x  A . 

INPUT  obviously  endorses  the  work  it  has  done  (and  is  doing)  in 
productivity  improvement.    Specifically,  we  believe  the  following: 

It  is  necessary  to  understand  IBMs  software  directions  because,  for 
better  or  for  worse,  IBM  is  creating  the  systems  software 
environment  within  which  other  software  systems  must  be  integrated. 
Of  particular  importance  are  SNA,  Operating  Systems,  and  Data  Base 
Management  Systems.  All  of  these  areas  have  been  explored  in 
detail,  in  past  INPUT  reports,  and  this  year  special  emphasis  is 
being  given  to  Operating  Systems  at  all  levels  in  the  processing 
h  i  erarchy . 
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We  believe  that  integration  of  multiple  languages,  operating 
systems,  and  data  bases  systems  is  inevitable  and  there  is  a 
tremendrous  need  -for  a  power-ful  me  ta-1  anguage(s)  -for  concise 
communication  across  hardware-software  systems,  program  modules, 
user  languages,  and  data  base  management  systems.  It  is  strongly 
recommended  that  APL  be  considered  as  a  possible  tool  -for  this 
purpose . 

We  also  believe  that  performance  at  the  hardware-so-f tware  level  is 
becoming  and  increasing  critical  issue,  and  it  is  not  possible  to 
depend  on  advances  in  computer  technology  to  compensate  -for  the 
increasingly  complex  systems  which  are  being  developed.  It  is  well 
known  that  there  are  limits  on  computer  processing  power,  regardless 
o-f  cost,  and  that  o-f  these  limitations  can  be  exceeded  by  many  o-f 
the  algorithms  o-f  both  artificial  intelligence  and  operations 
research.  There  is  great  need  to  be  able  to  predict,  monitor  and 
control  hardware-so-f  tware  performance   if  these  limits  are  not 

to  be  exceeded  (and  unworkable  systems  developed). 

It  is  necessary  to  become  familiar  with  the  tools  of  AI  and  OR  as 
both  possible  solutions  to  the  productivity  problem  and  as 
potentially  being  part  of  the  problem.  We  strongly  recommend  those 
responsible  for  complex  software  development  become  familiar  with 
such  important  subjects  as  Game  Theory,  Information  Theory, 
Decision  Theory,  and  Queuing  Networks  etc.  before  proceeding  with 
the  development  of  Decision  Support  and  Expert  Systems  which  may 
fail  with  disastrous  and  even  catastrophic  consequences. 
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o  In  summary,  we  recommend  that  the  productivity  pyramid  be  understood  and 
used  as  the  -foundation  -for  the  development  of  a  productivity  improvement 
program,  and  that  the  effectiveness  o-f  any  productivity  improvement 
program  be  measured  against  the  -four  performance  levels  established  in 
this  report  (hardware-software,  human-machine  dyad,  work  unit,  and 
i  nst  i  tu  t  i  onal ) 


-46- 


INPUT 


